{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d1f20875-c50d-46df-906b-49ec1d0c6002",
   "metadata": {},
   "source": [
    "# 8.2 使用指数衰减学习率训练模型\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c90a18b9-d8e8-431c-856c-35f1e4ed05e1",
   "metadata": {},
   "source": [
    "### 1.任务描述\n",
    "\n",
    "某损失函数的公式为：$Loss=(w+1)^2$\n",
    "\n",
    "要求：将参数w初始化为5，初始学习率LR_BASE设置为0.2，学习率衰减指数LR_DECAY设置为0.99，每多少轮衰减一次学习率LR_STEP设置为1，迭代30次，使用梯度下降法优化w参数，求Loss的最小值。\n",
    "\n",
    "打印迭代过程中的学习率的变化。"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f5b4fc39-cbcf-432a-bf1e-e75e642d4b87",
   "metadata": {},
   "source": [
    "### 2.知识准备\n",
    "\n",
    "见教程。\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b624ebee-980f-4c1e-b963-a24ff0b669f6",
   "metadata": {},
   "source": [
    "### 3.任务分析\n",
    "\n",
    "通过人为设定超参数可以实现学习率衰减指数的动态变化。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "435c6090-cfda-4f46-a550-22a368e41e4a",
   "metadata": {},
   "source": [
    "### 4.任务实施\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ec75eb6c-5da3-467d-a471-ca3b47242dd6",
   "metadata": {},
   "source": [
    "执行代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2ae9da58-e339-4d22-9f8d-ca255711d89e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "After 5 epoch,w is -0.485653,loss is 0.689104,lr is 0.190198\n",
      "After 10 epoch,w is -0.948612,loss is 0.006483,lr is 0.180876\n",
      "After 15 epoch,w is -0.994086,loss is 0.000081,lr is 0.172012\n",
      "After 20 epoch,w is -0.999224,loss is 0.000001,lr is 0.163581\n",
      "After 25 epoch,w is -0.999885,loss is 0.000000,lr is 0.155564\n",
      "After 30 epoch,w is -0.999981,loss is 0.000000,lr is 0.147940\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "# 1，参数初始化为5\n",
    "w = tf.Variable(tf.constant(5, dtype=tf.float32))\n",
    "# 2，超参数\n",
    "# 迭代次数\n",
    "epoch = 30\n",
    "# 初始学习率\n",
    "LR_BASE = 0.2\n",
    "# 学习率衰减指数\n",
    "LR_DECAY=0.99\n",
    "# 多少轮衰减一次学习率\n",
    "LR_STEP=1\n",
    "# 定义顶层循环，表示对数据集循环epoch次，迭代40次\n",
    "for epoch in range(1,epoch+1):\n",
    "    # 计算学习率，根据当前迭代次数，动态改变学习率的值\n",
    "    lr = LR_BASE * LR_DECAY ** (epoch / LR_STEP)\n",
    "    with tf.GradientTape() as tape:\n",
    "        # 损失函数\n",
    "        loss = tf.square(w + 1)\n",
    "    #求导\n",
    "    grads = tape.gradient(loss, w)  \n",
    "    # 参数更新\n",
    "    w.assign_sub(lr * grads)\n",
    "    if epoch%5==0:\n",
    "        print(\"After %s epoch,w is %f,loss is %f,lr is %f\" % (epoch, w.numpy(), loss,lr))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3b3e36e-8418-4caf-af51-d8ac80e2a321",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
